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Within the rephca framework we study analytically the in- 
stance space of the number partitioning problem. This clas- 
sic integer programming problem consists of partitioning a 
sequence of A'^ positive real numbers {ai, 02, ... , ffljv} (the in- 
stance) into two sets such that the absolute value of the dif- 
ference of the sums of aj over the two sets is minimized. We 
show that there is an upper bound a^N to the number of 
perfect partitions (i.e. partitions for which that difference 
is zero) and characterize the statistical properties of the in- 
stances for which those partitions exist. In particular, in the 
case that the two sets have the same cardinality (balanced 
partitions) we find ac = 1/2. Moreover, we show that the 
disordered model resulting from the instance space approach 
can be viewed as a model of replicators where the random 
interactions are given by the Hebb rule. 
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I. INTRODUCTION 

Most statistical mechanics analyses of combinatorial 
optimization problems have concentrated on the charac- 
terization of average properties of minima of a given cost 
function [|]j|]. Usually, the cost function depends on a 
large set of fixed parameters, termed the instance of the 
optimization problem [e.g. the distances between cities in 
the celebrated travelling salesman problem (TSP)] which, 
in the framework of statistical mechanics, are treated as 
quenched random variables distributed according to some 
known probability distribution. Furthermore, in order to 
consider the subspace of configurations with a given av- 
erage cost, one defines a probability distribution on the 
space of configurations (e.g. the NI/2N different tours 
or ordering of the cities in the TSP), namely, the Gibbs 
distribution with 'temperature' T — 1/13. The zero- 
temperature limit then singles out the configurations that 
minimize the cost function (ground states). Clearly, in 
this formulation the configurations are treated as fast, 
annealed variables. 

Instead, in this work we explore the opposite view- 
point, namely, given a set of configurations we want to 
characterize the subspace of instances for which those 
configurations have a certain cost. This approach may 
be viewed as a best-case analysis in the sense that one 
searches for particularly easy instances that fit the given 
solutions. The situation here is similar to the physics 
approach to neural networks. In a first stage, atten- 
tion was given to the neural dynamics while the coupling 
strengths between neurons were kept fixed according to 



some variant of the Hebb learning rule [g||^ . (The neural 
dynamics itself can be viewed as a versatile heuristic in 
which the optimization problem is embedded in the neu- 
ral couplings [p[.) In the second stage which followed the 
seminal work of Gardner Q , the focus was on the charac- 
terization of the couplings distribution that ensures the 
stability of a given set of neural states. Gardner's formu- 
lation allowed a rich interchange of concepts and meth- 
ods between the statistical physics and the computational 
learning theory communities 0] . 

The specific optimization problem we consider in this 
paper is the number partition problem (NPP) |g|,y which 
has received considerable attention in the physics liter- 
ature recently flQ-p^. It is stated as follows. Given a 
sequence of N positive real numbers a = {oi, 02, ... , a^} 
(the instance) , the NPP consists of partitioning them into 
two disjoint sets Ai and A2 such that the difference 
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is minimized. Alternatively, we can search for the Ising 
spin configurations s = {si, . . . , sn} that minimize the 
cost function 
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E (s) = I > fliS 
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where Sj = 1 if aj G Ai and Sj ~ —1 if aj G A2- Despite 
its simplicity, the NPP was shown to belong to the NP- 
complete class, which basically means that there is no 
known deterministic algorithm guaranteed to solve all 
instances of this problem within a polynomial time bound 

In the proposed framework, we aim at characterizing 
the subspace of instances {a} for which the fixed set of 
partitions {s'} I = 1, ..., P, are perfect, i.e., E (s') = V/. 
To achieve this we define the energy in the instance space 
as 
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so that the P partitions are perfect only if 7i = 0. Hence- 
forth we will assume that P increases linearly with TV, 
i.e., P = aN. Furthermore, we assume that the compo- 
nents s\ are statistically independent random variables 



drawn from the probability distribution 
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where the weights of the Dirac delta functions are chosen where 



so that (s ) = m/w N . The motivation for this choice is 
twofold. First, the exhaustive search in the Ising configu- 
ration space for TV < 26 as well as the analytical solution 
of the linear relaxation of the NPP indicate that the av- 
erage difference between the cardinalities of sets Ai and 
A2, 
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vanishes like 1/yN for large N [g^. Second, this scaling 
yields a non-trivial thermodynamic limit, N ~^ 00, for 
the average free-energy density associated to the Hamil- 
tonian (||). 

In this paper we will apply standard statistical me- 
chanics techniques to study analytically the ground states 
of the Hamiltonian (H). We concentrate our analysis on 
the zero-energy instances (i.e., instances for which perfect 
partitions exist) only, since the properties of the non-zero 
energy instances depend strongly on the rather arbitrary 
choice of the energy (bI). Moreover, perfect partitions 
are important from a practical viewpoint as they may 
have code-breaking implications |l3] and so it may be 
of interest to estimate the maximum number of perfect 
partitions that can be encoded in an arbitrary instance, 
as well as to characterize those instances that maximize 
the number of coded perfect partitions. 

The rest of this paper is organized in the following 
way. In Sec. |^ we use the replica method to evaluate 
the average free-energy density in the thermodynamic 
limit and to derive the replica-symmetric order param- 
eters that describe the statistical properties of the in- 
stance space. In particular, we show that there is a crit- 
ical value, ttc (m) N , which limits the number of perfect 
partitions. Also in that section, we study the stability 
of the replica-symmetric solution against replica symme- 
try breaking and show that the zero-energy instances can 
reliably be described by the replica-symmetric order pa- 
rameters. In Sec. [II we calculate the probability den- 
sity for a given entry, say a^, to have value a. This is 
achieved by integrating the joint probability distribution 
(the Gibbs distribution) over all entries except Ofe. Fi- 
nally, in Sec. [V we present some concluding remarks. In 
particular we show that the disordered model considered 
here is formally identical to a model of replicators with 
the random interactions given by the Hebb rule. 
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is the partition function and j3 = 1/T is the inverse tem- 
perature. Taking the limit T ^ in Eq. (1^) ensures that 
only the instances that minimize 7i (a) will contribute to 
Z . Here (. . .) stands for the average over the partitions 
s' [l = 1,...,P). The constraint on the mean of the 
instance vector is needed in order to exclude the trivial 
solution a = 0. Fortunately, the arbitrary parameter R 
does not play any relevant role in the theory, giving only 
the scale of the order parameters of the model. 

As usual, the quenched average in Eq. (g) is evaluated 
through the replica method: using the identity 



(InZ) = lim iln(Z") 
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we first evaluate {Z^) for integer n and then analytically 
continue to n = 0. Using standard techniques [ [L4[ we 
obtain, in the thermodynamic limit 
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where 
Gi = 
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and 
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II. REPLICA APPROACH 

Following the standard prescription of performing 
quenched averages on extensive quantities only g], we 
define the average free-energy density / as 
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The extremum in Eq. (E 
rameters [qpf, ,Rp,Qp,qpS,( 
rameters 



qpS 



is taken over all order pa- 
Thc physical order pa- 
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measure the overlap between a pair of different equilib- 
rium instances a^ and a , and the overlap of an equilib- 
rium instance a'' with itself, respectively. Here, (. . .)t 
stands for a thermal average. 



A. Replica-symmetric solution 
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The replica-symmetric average energy density e^s = 
d iP frs) /df3 is given by 



Crs/aR^ = 
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[1 + 2/3' (Q - g)]' 1 + 2/3' (Q - q) 
which vanishes in the limit f3' ^ oo provided that q < Q 



To proceed further we make the replica symmetric 
ansatz, i.e., we assume that the values of the order pa- 
rameters are independent of their replica indices 
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Evaluation of Eqns. ( pl)| ) and (11) with this ansatz is 
straightforward. In order to write the replica symmetric 
average free-energy density it is convenient to introduce 
the new variables 



R 



2[Q + q 



ri = R-^ 



and rescale the temperature /3' = /3i?^ so that 
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is the Gaussian measure. Thus it is clear from Eq. ( p^ ) 
that the parameter R yields the scales of the temperature 
and free-energy, not affecting in any significant way the 
physical, replica-symmetric order parameters 
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and 




FIG. 1. Average variance of the zero-energy instance en- 
tries Q — g as a function of a for m — 0, 0.5, 1, 1.5, 2, and 
3. The value of a at which the variance vanishes(ac) gives an 
upper bound to the number of perfect partitions. 
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FIG. 2. Average overlap between two different zero-energy 
instances q as a function of a for m — 0, 0.5, 1, 1.5, 2, and 3. 
The curves end at a = Oc- 



As justified in Sec. | we will focus on this limit only. 
After some algebra, the saddle-point equations in this 
limit are written as 
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Here Qc stands for the order parameter Q evaluated at 
ac- For m — Q we can solve these equations analytically: 
we find that A vanishes like m? /\/2t: and so a^ — 
and Qc = '^- 



1/2 



■n = 
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with Sz given by Eq. (Uj). In general these equations can 
be solved numerically only. In Figs. ^ and y we present 
the dependence of Q — q and q, respectively, on a for 
different values of m. For a = we find Q — q = q = 
1, 6 — {I + m^)/2, and ry = 0, while r diverges like 
l/v2a. According to the physical meaning of the order 
parameters given in Eqns. (II^) and (pO|), the difference 
Q — q measures the average variance of the zero-energy 
instance entries: the larger this difference, the larger the 
dispersion of the instance entries. Interestingly, for fixed 
TO > this variance reaches its maximum for a > 0. The 
divergence of the order parameters Q and q (and of their 
difference, as well), for m —^ oo and a ^ is expected, 
since in order that an extremely unbalanced partition 
become a perfect partition there must exist some very 
large entries to compensate for the much larger number 
of entries in one of the sets. Moreover, we observe from 
Fig. m that for fixed m there is a value of a = etc at which 
the overlap between two zero-energy instances q equals 
its maximal value Q. This results signals the shrinking of 
the zero-energy instance subspace to instances differing 
from a microscopic number of entries aj only. Besides, 
it gives the limit of existence of the solutions with zero- 
energy: for a > ac there are no zero-energy instances. 
Taking the limit q ^>- Q in the saddle-point equations (|2^ 
- M, yields 
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FIG. 3. Instance independent upper bound to the number 
of perfect partitions a^ as a function of the parameter m 
which measures the unbalance of the partitions. For balanced 
partitions (m — 0) we find Oc ~ 1/2. 
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FIG. 4. Average overlap of a zero-energy instance with it- 
self calculated at a^ as a function of m. For balanced parti- 
tions {m — 0) we find Qc = tt. 

In Figs. H and ^ we show ac and Qc, respectively, as 
functions of m. The dependence of ac on m indicates 
that instances for which there are an extensive number 
of unbalanced perfect partitions become very rare with 
increasing to. In particular, there are no zero-energy in- 



stances for partitions with average cardinalities difference 
[see Eq. (^] of order of 1. 



B. Stability analysis 

The condition for the local stability of the replica- 
symmetric saddle-point is given by [Q 
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where 71 and 72 are the transverse eigenvalues ||T5| of the 
matrices of second derivatives of Gi and G2 with respect 
to Qps and Qps, respectively, evaluated at the replica- 
symmetric saddle-point. After some algebra wc find that 
condition (ROh reduces to 
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where 
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Taking the limit q —f Q we can easily show that 

r]{Q- q) ^ ac (33) 

and 
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with ac given by Eq. (p^, so that the left hand side of 
Eq. (|3l| ) equals 1 at a = ac. Moreover, we have ver- 
ified numerically that this stability condition is always 
satisfied for a < Ur- 



III. PROBABILITY DISTRIBUTION OF 
ENTRIES 



The traditional probabilistic approach to study opti- 
mization problems introduces a probability distribution 
over the space of instances. The main objection to this 
approach is that one rarely knows what probability distri- 
bution is realistic. In the NPP, for instance, it is usually 
assumed that the entries a^ are statistically independent 
random variables distributed uniformly in the unit inter- 
val |^-[lC[|. In this section we calculate analytically the 
distribution of probability that a certain entry, say ak , of 
a zero-energy instance assumes the value a, defined by 

/ 1 1"°° 
Vk (a) = lim ( {5{ak -a))j, )= lim ( - / \\daj 

M^-^E«^) Hak-a)e-^'''^A (35) 



where Z and Ti are given by Eqns. (0) and (||), re- 
spectively. As all entries are equivalent we can write 
Vk (a) = V (a) V/c. Hence to evaluate Eq. ( [35| ) we intro- 
duce the auxiliary energy 
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h'^S{ak- a) . 
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so that 
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where Zaux is the partition function (M) with Ti replaced 
by Tlaux- Of course, we note that the entries (oi, . . . , ajv) 
are not statistically independent and their joint probabil- 
ity distribution is simply the Gibbs probability distribu- 
tion 



W (a) = i exp [-PH (a)] 



(38) 



As expected, Eq. ( p5| ) is recovered by integrating this 
joint distribution over Uj for all j ^ k and then setting 
Uk = a. Using Eq. ( p7| ) the calculations needed to eval- 
uate V (a) become analogous to those used in the evalu- 
ation of the free-energy density (|^). Within the replica- 
symmetric framework and in the limit /3 — > 00 with q < Q 
the final result is 
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which for a = reduces to 



7'(a) = iexp(-^) a>0. 
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To handle a possible singularity in the limit q ^ Q it 
is more convenient to consider instead the cumulative 
distribution function defined by 
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Taking the limit q —^ Q yields 



, {rj/2R^) 



1/2 



erfc ^2 



Cr (a) = 1 erfc 



A 






Rm? 



(41) 



(42) 



where ac and A are given by Eqns. ( |27l ) and (|29|), respec- 
tively. The interesting feature of this distribution is that 
Cc (0) is non-zero indicating thus that the probability dis- 
tribution (B9[) evaluated at a = ac has a delta peak in 
a = 0. Explicitly, 



Vc (a) = Cc (0) 5[a) + -^ a > 0, (43) 

da 



which for m = Q reduces to 



exp 



In Fig. |g we show the cumulative distribution function 
C{a) for TO = and several values of a. We note that 
C(0) ^ only at a = a^- 
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FIG. 5. Cumulative distribution function of 

the zero-energy instance entries for m = and a = 0, 0.4, 
and 0.5. Note that C(0) / only at a = Oc = 0.5. 



IV. CONCLUSION 

Since the instance entries are continuous variables ai G 
[0,oo) Vi we can resort to a simple gradient descent al- 
gorithm to find the minima of the energy 7i (a) , defined 
by Eq. (0), which have a given mean. More pointedly, it 
can be easily verified that the dynamics 
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minimizes Ti. (a) while the mean ^^ at is a constant of 
motion. Interestingly, Eq. ( |4q ) is readily recognized as a 
particular realization of the classical replicator equation 
which has been used to describe the evolution of self- 
reproducing entities (replicators) in a variety of fields, 
such as game theory, prebiotic evolution and sociobiol- 
ogy, to name only a few |I^. In fact, Ui can be viewed 
as the concentration of species i whose fitness Hi is the 
derivative Hi = dH/dai of a fitness functional H(a). 
The (infinite) population of replicators is composed of 
N different species which evolve under the constraint of 



constant total concentration. The disordered model con- 
sidered here is a variant of the model of replicators with 
random interactions studied by Diederich and Opper |1^] 
(see also [Q) in which the fitness functional is given by 



Hdo (a) = ^ Jr 
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where the couplings Jy = Jji {i ^ j) are independent, 
identically distributed Gaussian random variables with 
mean zero and variance 1/N ^ while the self-interactions 
are non-random, species independent control parame- 
ters of the model, i.e., Ja = u \fi. Clearly, the en- 
ergy given by Eq. (^ can be rewritten in the form of 
Eq. (^6|) with the couplings given by the Hebb rule 

Jij ~ TfJ^i ^i-^^j [i|- We note that in our model the 
self- interaction is Ja — a ^i, while the mean and vari- 
ance of the off diagonal couplings are am^/N -^ and 
a (l — TO^/A^^) /N -^ ct/N, respectively. Moreover, as 
in the Hopfield model ^ , though the s' are independent 
random variables, the couplings Jij are not. It is in- 
teresting thus to interpret our results in the light of the 
random replicator model: for a < ac the global optimum 
of the fitness functional, H (a) — 0, can be reached with 
the coexistence of all species; for a — ac reaching that 
optimum requires the extinction of a macroscopic num- 
ber of species, as signaled by the delta peaks in a = 0; 
and for a > ac the interactions between species are such 
that the optimum is never reached. 

To conclude, we mention that while the traditional ap- 
proach of Computer Science to the validation of combi- 
natorial search algorithms focuses almost exclusively on 
the instance space (e.g. the worst-case analysis is basi- 
cally a search for instances that give the poorest perfor- 
mance of the algorithm under study [Q), the statistical 
mechanics approach has concentrated mainly on the con- 
figuration space, with the instances being drawn from 
arbitrary probability distributions H]. Building on the 
work of Gardner on neural networks ra], we illustrate in 
this paper the usefulness of equilibrium statistical me- 
chanics tools to investigate the statistical properties of 
the instance space as well. For the optimization prob- 
lem we have considered, namely, the number partition- 
ing problem, we have searched the instance space for the 
best (easiest) instances to show that there is a maximum 
number of uncorrelated perfect partitions, ac (to,) N (see 
Fig. g). In particular, for balanced partitions (m = 0) 
we find ac{0) = 1/2. Clearly, this result yields an upper 
bound to the number of perfect partitions (ground-state 
degeneracy) that can be found for any arbitrary instance. 
As in the neural networks case, the instance space anal- 
ysis proposed in this paper can be extended to virtually 
all optinnzation problems. 
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